PageRank Pipeline Benchmark: Proposal for a Holistic System Benchmark for Big-Data Platforms

摘要

The rise of big data systems has created a need for benchmarks to measure andcompare the capabilities of these systems. Big data benchmarks present uniquescalability challenges. The supercomputing community has wrestled with thesechallenges for decades and developed methodologies for creating rigorousscalable benchmarks (e.g., HPC Challenge). The proposed PageRank pipelinebenchmark employs supercomputing benchmarking methodologies to create ascalable benchmark that is reflective of many real-world big data processingsystems. The PageRank pipeline benchmark builds on existing prior scalablebenchmarks (Graph500, Sort, and PageRank) to create a holistic benchmark withmultiple integrated kernels that can be run together or independently. Eachkernel is well defined mathematically and can be implemented in any programmingenvironment. The linear algebraic nature of PageRank makes it well suited tobeing implemented using the GraphBLAS standard. The computations are simpleenough that performance predictions can be made based on simple computinghardware models. The surrounding kernels provide the context for each kernelthat allows rigorous definition of both the input and the output for eachkernel. Furthermore, since the proposed PageRank pipeline benchmark is scalablein both problem size and hardware, it can be used to measure and quantitativelycompare a wide range of present day and future systems. Serial implementationsin C++, Python, Python with Pandas, Matlab, Octave, and Julia have beenimplemented and their single threaded performance has been measured.